User experienced remote monitoring of a communications network

ABSTRACT

Various systems, computer program products, and methods are disclosed for user experienced remote monitoring of a communications network. For example, a method may include receiving air interface information from a mobile device. The air interface information may originate from signals communicated between the mobile device and the communications network. The air interface information may indicate an actual communication quality between the mobile device and the communications network. The method may further include receiving user experience information. The user experience information may indicate a perceived communication quality at the mobile device. The method may further include associating, by the server device, the air interface information and the user experience information with one another. The method may further include determining, by the server device, whether the perceived communication quality is due to the actual communication quality based on the associated air interface information and user experience information.

FIELD OF THE INVENTION

The disclosure relates to remote monitoring of a communications networkand in particular to user experienced and detailed air interface remotemonitoring of a communications network.

BACKGROUND OF THE INVENTION

Conventional network communications monitors do not indicate“real-world” or production use of mobile devices. Such conventionalsystems typically require specialized equipment that actively testsparticular portions of the communications network. However, thesesystems do not capture everyday use of the communications network bymobile devices. Thus, only an incomplete understanding of theperformance of a communications network may be achieved withconventional systems. Other conventional systems use only basiccommunication quality parameters that indicate perceived communicationquality at mobile devices. For example, a signal strength meteravailable from operating systems of mobile devices may indicate aperceived quality at mobile device based on basic quality parameters.However, for various reasons, these basic indications fail to indicatethe true health or quality of the underlying network.

What is needed is remote monitoring of a communications network based ondetailed air interface information that is exchanged between productionmobile devices and the communications network and that indicates actualperformance of the communications network. What is further needed isremote monitoring of a communications network based on user experienceinformation that indicates a quality of performance experienced at amobile device in order to determine whether the communications networkis responsible for the quality of performance. These and other problemsexist.

SUMMARY OF THE INVENTION

Various systems, computer program products, and methods are disclosedfor user experienced remote monitoring of a communications network.According to various implementations of the invention, the method mayinclude receiving, by a server device, air interface information from amobile device, wherein the air interface information originates fromsignals communicated between a mobile device and the communicationsnetwork, and wherein the air interface information indicates an actualcommunication quality between the mobile device and the communicationsnetwork. The method may further include receiving, by the server device,user experience information. The user experience information indicates aperceived communication quality at the mobile device. The method mayfurther include associating, by the server device, the air interfaceinformation and the user experience information with one another. Themethod may further include determining, by the server device, whetherthe perceived communication quality is due to the actual communicationquality based on the associated air interface information and userexperience information.

According to various implementations of the invention, a method ofuser-experienced remote monitoring of a communications network mayinclude accessing, by a mobile device, one or more chipsets of themobile device. The method may include collecting, by the mobile device,air interface information from signals communicated between the mobiledevice and the communications network via the accessed one or morechipsets. The air interface information may indicate an actualcommunication quality. The method may further include determining, bythe mobile device, user experience information. The user experienceinformation may indicate a perceived communication quality at the mobiledevice. The method may further include communicating, by the mobiledevice, the air interface information and the user experienceinformation to a remote device. The air interface information and theuser experience information are used to determine whether the perceivedcommunication quality is caused by the actual communication quality.

According to various implementations of the invention, the method mayinclude receiving, by a server device, air interface information from amobile device, wherein the air interface information originates fromsignals communicated between a mobile device and the communicationsnetwork, and wherein the air interface information indicates an actualcommunication quality between the mobile device and the communicationsnetwork. The method may further include receiving, by the server device,user experience information. The user experience information indicates aperceived communication quality at the mobile device. The method mayfurther include associating, by the server device, the air interfaceinformation and the user experience information with one another. Themethod may further include storing, by the server device, the associatedair interface information and user experience information.

According to various implementations of the invention, the method mayinclude receiving, by a server device, air interface information from amobile device, wherein the air interface information originates fromsignals communicated between a mobile device and the communicationsnetwork, and wherein the air interface information indicates an actualcommunication quality between the mobile device and the communicationsnetwork. The method may further include receiving, by the server device,parametric user experience information and subjective user experienceinformation. The parametric and subjective user experience informationindicates a perceived communication quality at the mobile device. Themethod may further include associating, by the server device, the airinterface information and the user experience information with oneanother. The method may further include determining, by the serverdevice, whether the perceived communication quality is due to the actualcommunication quality based on the associated air interface informationand user experience information.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated into and constitute apart of this specification, illustrate one or more examples ofimplementations of the invention and, together with the description,serve to explain various principles and aspects of the invention.

FIG. 1 is a block diagram illustrating a system of user-experiencedremote monitoring of a communications network, according to variousimplementations of the invention.

FIG. 2 is a block diagram illustrating examples of components of aserver and a mobile device for user-experienced remote monitoring of acommunications network, according to various implementations of theinvention.

FIG. 3 is a block diagram illustrating an example of a mobile device foruser-experienced remote monitoring of a communications network,according to various implementations of the invention.

FIG. 4 is a flow diagram illustrating an example of a process ofuser-experienced remote monitoring of a communications network at aserver, according to various implementations of the invention.

FIG. 5 is a flow diagram illustrating an example of a process ofuser-experienced remote monitoring of a communications network at amobile device, according to various implementations of the invention.

FIG. 6 is a flow diagram illustrating an example of a process ofuser-experienced remote monitoring of a communications network,according to various implementations of the invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a block diagram illustrating a system 100 of user-experiencedremote monitoring of a communications network 106, according to variousimplementations of the invention. According to various implementationsof the invention, system 100 may include, for example, mobile devices102 (illustrated in FIG. 1 as mobile device 102A, 102B, . . . , 102N),base stations 104 (illustrated in FIG. 1 as base station 104A, 104B, . .. , 104N), a communications network 106, and a server 110. In someimplementations, mobile devices 102, base stations 104, and server 110may be communicably coupled to one another via communications network106, which may be at least partially formed from base stations 104.Communications network 106 may include a Local Area Network, a Wide AreaNetwork, a cellular communications network, a Public Switched TelephoneNetwork, and/or other network or combination of networks.

According to various implementations of the invention, system 100 maydetermine a quality of network communication measured or perceived atmobile device 102 based on user experience information and determinewhether the measured or perceived quality was due to actual qualitybased on air interface information. In some implementations, the userexperience information and the air interface information may beassociated with a device identifier and/or time identifier so that theymay be correlated with one another. Thus, actual data from users ofmobile devices may be logged and analyzed rather than attempting torecreate a reported problem or using different information fromdifferent mobile devices to attempt to assess communication quality.

In some instances, the actual network performance may not cause thenetwork performance measured or perceived at mobile device 102. Forexample, a network performance measured or perceived at mobile device102 such as a low signal strength indication may result from a faultyantenna of mobile device 102 rather than actual poor performance ofcommunications network 106. In some implementations, in order to assesswhether the measured or perceived quality is due to actual quality ofcommunications network 106, server 110 may associate air interfaceinformation that is accessed from one or more chipsets of the mobiledevice 102. In some implementations, the air interface information mayinclude, for example, air interface protocols, messages, parametersreported to and from mobile device 102 and a base station 104 to controlnetwork traffic, network configurations, and/or other informationsignaled between mobile device 102 and communications network 106. Insome implementations, the signaling may include UMTS, CDMA, LTE, and/orother communication technologies. The air interface information iscommunicated between the mobile device 102 and a base station 104 ofcommunications network 106 via the one or more chipsets. Thus, byaccessing information at a chipset level, information communicatedbetween mobile device 102 and base station 104 that indicates actualnetwork quality may be used to determine whether communications network106 was responsible for the network quality measured or perceived atmobile device 102 as indicated by the user experience information. Inthis manner, actual quality of communications network 106 may beassessed based on information not generally used by conventionalcommunications network monitors.

In some implementations, mobile devices 102 may include production-levelmobile devices operating on a communications network such as a cellularcommunications network provided by a cellular service provider. Forexample, mobile devices 102 may include a laptop computer, a tabletcomputer, a cellular phone, or other device configured to operate oncommunications network 106. In some implementations, users may operatemobile devices 102 in a production environment. In other words, mobiledevices 102 may be operated by users for personal or business purposesduring which air interface information and/or user experienceinformation may be monitored or otherwise collected. In this manner,system 100 may remotely monitor mobile devices 102 and communicationsnetwork 106 during the course of production (everyday or otherwisenormal) use of mobile devices 102. In some implementations, users mayoperate mobile devices 102 in a testing environment. In other words,mobile devices 102 may be operated by users during testing proceduresthat assess the quality of communications network 106.

In some implementations, server 110 may receive air interfaceinformation from mobile device 102. In some implementations, the airinterface information originates from signals communicated betweenmobile device 102 and communications network 106. For example, the airinterface information may be exchanged between mobile device 102 andbase station 104 while mobile device 102 is in communication withcommunications network 106. In some implementations, the air interfaceinformation indicates a communication quality between mobile device 102and communications network 106. “Communication quality” (also usedinterchangeably herein with “quality of network communication”) refersto a quality of network communication during an actual communication,such as a phone call or data transmission, and quality while mobiledevice 102 is not being used during an actual communication. In otherwords, so long as mobile device 102 is communicably coupled tocommunications network 106 and is able to execute an actualcommunication, a communication quality may be assessed. “Communicationquality” may be used interchangeably herein with “quality of networkcommunication.”

In some implementations, server 110 may receive user experienceinformation. In some implementations, the user experience informationindicates a quality of performance perceived at mobile device 102. Insome implementations, server 110 may determine a quality of performanceperceived at mobile device 102 based on the user experience information.

In some implementations, the user experience information may includeparametric user experience information, which may include basicindicators of network performance such as, for example, a signalstrength, a battery level, and/or other basic indicators of networkperformance. In some implementations, the parametric user experienceinformation may be accessed via a public Applications ProgrammingInterface of an operating system of mobile device 102. In someimplementations, the parametric user experience information may bederived or otherwise determined from air interface information. In theseimplementations, air interface information may be used to generate aparameter that scores the user experience. For example, information froma TCP stack together with air interface information may be used togenerate a data performance quality index in order to assess a user'sexperience. In another example, a speech quality index may be generatedbased on air interface information to assess a user's experience.

In some implementations, the user experience information may includesubjective user experience information, non-limiting examples of whichmay include a user indication of: a call being dropped, a call havingpoor voice quality, slow data transmission rates, and/or other userindications of network quality. In some implementations, the subjectiveuser experience information may include one or more user ratings and/orresponses to questionnaires associated with the measured/perceivedquality. Such ratings may be numeric such as a rating on a scale of 0-5of a quality of a phone call, descriptive such as “poor,” “belowaverage,” “average,” “good,” “excellent” and so forth, and/or otherratings that reflect a user's opinion of the quality of communication atmobile device 102. In some implementations, the subjective userexperience information may include responses to one or more questionsposed by mobile device 102 to elicit the subjective user experienceinformation. Thus, according to various implementations of theinvention, the user experience information may indicate parametric(i.e., measured) or subjective (i.e., user opinion) quality of networkperformance perceived at mobile device 102. In some implementations, theuser experience information may be used to determine which data sets toanalyze. For example, server 110 may filter out some user experienceinformation but not others. In this manner, only particular (such aspoor) communication quality indicated by the user experience informationmay be analyzed.

In some implementations, server 110 may determine whether the perceivedquality of performance occurred due to network quality based on the airinterface information. In some implementations, server 110 may determinethat communications network 106 did not cause the perceived networkquality indicated by the user experience information. In otherimplementations, server 110 may determine that communications network106 caused the perceived network quality. In these implementations,server 110 may drill-down to determine the root cause based on the airinterface information.

In some implementations, server 110 may generate a report based on theair interface information and the user experience information. In someimplementations, the report indicates or may be used to indicate whetherthe actual quality of communications network 106 was the cause of thenetwork performance measured or perceived at mobile device 102. In someimplementations, the report may be accessible by devices coupled tocommunications network 106. For example, the report may be communicatedvia web pages, reports directly communicated to mobile device 102,email, etc.

In some implementations, by determining the cause of the quality ofnetwork performance, server 110 may facilitate changing communicationsnetwork 106 in order to address the quality. For example, server 110 mayfacilitate addition of new base stations 104, reconfiguration ofexisting base stations 104, and/or other actions that change orotherwise alter the infrastructure/configuration of communicationnetwork 106.

In some implementations, server 110 may periodically receive airinterface information and user experience information. In someimplementations, server 110 may store the information for lateranalysis. In some implementations, server 110 may correlate the receivedair interface information and user experience information with oneanother. For example, server 110 may timestamp at least one of the airinterface information and user experience information and/or may usetimestamps already recorded with one or both. In other implementations,server 110 may receive the air interface information and user experienceinformation already correlated.

In some implementations, server 110 may determine whether the quality ofperformance occurred due to network quality based on the air interfaceinformation periodically at intervals. For example, server 110 mayperiodically review the received user experience information todetermine a perceived quality of network performance at various times.

In some implementations, server 110 reviews user experience informationthat indicates a quality that does not exceed a predefined level. Inthis manner, server 110 may periodically analyze instances in which userexperience information indicates certain levels of network performance.

For example, in some implementations, server 110 may periodically reviewuser experience information that reflects user indications of, forexample, “poor” or “below average” quality of network performance atmobile device 102. In these implementations, server 110 may analyze userexperience information to determine causes of below average or poorindications of communication quality.

In some implementations, server 110 may periodically review userexperience information that reflects user indications of, for example,“good” or “excellent” quality of performance at mobile device 102. Inthese implementations, server 110 may analyze user experienceinformation to determine causes of good or excellent indications ofcommunication quality.

In some implementations, server 110 may receive a triggering event thatcauses server 110 to determine whether the quality measured or perceivedat mobile device 102 is caused by communication network 106. Thetriggering event may include, for example, a user indication thatquality is at a particular level, a determination by mobile device 102that quality is at a certain level, a determination by communicationsnetwork 106 that quality is at a particular level, a determination thatmobile device 102 is within a geographic location (or particulardistance to a base station 104), and/or other event that causes server110 to determine whether the quality measured or perceived at mobiledevice 102 is caused by communication network 106. In theseimplementations, server 110 may associate air interface information withthe triggering event in order to determine whether the actualperformance of communications network 106 caused a perceived qualityassociated with the triggering event.

In some implementations, server 110 may generate or otherwise determinework order commands to be performed by mobile device 102. The work ordercommands may include one or more instructions written in a programminglanguage, plain text instructions, compiled instructions, or otherinstructions that cause mobile device 102 to perform various tasks. Thevarious tasks may include, without limitation, when and what informationto collect and communicate to server 110, one or more questionnairesassociated with perceived quality to be posed to the user of mobiledevice 102, a quality ratings system to be presented to the user, and/orother instructions to be performed by mobile device 102 associated withuser-experienced remote monitoring.

In some implementations, mobile device 102 may receive the work ordercommands and perform various tasks. In some implementations, mobiledevice 102 may include an installed agent operating at mobile device102. In some implementations, the agent and/or work order commands maycause mobile device 102 to perform various tasks associated withuser-experienced remote monitoring.

In some implementations, mobile device 102 may access one or morechipsets of mobile device 102. In some implementations, mobile device102 may collect air interface information from signals communicatedbetween mobile device 102 and communications network 106 via theaccessed one or more chipsets. In some implementations, mobile device102 communicates the air interface information to communications network106. In some implementations, communications network 106 communicatesthe air interface information to mobile device 102.

In some implementations, mobile device 102 may determine user experienceinformation. In some implementations, mobile device 102 may communicatethe air interface information and the user experience information to aremote device such as server 110. In some implementations as discussedherein, server 110 may use the air interface information and the userexperience information to determine whether perceived communicationquality at mobile device 102 is caused by communications network 106.

In some implementations, mobile device 102 may generate a user interfaceconfigured to receive at least some of the user experience information.In some implementations, the user of mobile device 102 may use the userinterface in order to indicate the user's opinion that a particularlevel of quality was or is experienced at mobile device 102. Forexample, during or after a call, the user may use the user interface toindicate poor call quality. Thus, via the user interface, mobile device102 may receive an indication of a particular quality of communicationexperienced at mobile device 102. In some implementations, theparticular quality indicates a poor or otherwise undesirable level ofquality. In some implementations, the particular quality indicates agood or otherwise desirable level of quality.

In some implementations, the user interface may prompt the user toprovide responses to one or more questionnaires. For example, thequestionnaire may include questions such as, for example, “how would yourate the quality of your latest call?” or “how good was your datatransfer speed?” In some implementations, the responses may be in theform of one or more ratings of the quality such as a numeric,descriptive, or other ratings. In some implementations, the userresponses may include open-ended responses such as comments to beentered by the user. In this manner, via the user interface, mobiledevice 102 may receive one or more ratings from the user and/or otherresponses to questionnaires from the user.

In some implementations, the user initiates the user interface in orderto report or otherwise answer questionnaires associated with thecommunication quality. In some implementations, mobile device 102initiates the user interface based on an analysis of parametric userexperience information. For example, mobile device 102 may determine alow signal strength indication and in response, initiate the userinterface to collect subjective user experience information.

In some implementations, mobile device 102 communicates the airinterface information and the user experience information periodicallyat intervals. For example, mobile device 102 may collect the airinterface information and the user experience information periodically(such as every minute, every hour, every day, etc.). In someimplementations, mobile device 102 communicates the air interfaceinformation and the user experience information substantiallyimmediately after collecting the information. In some implementations,mobile device 102 stores the air interface information and the userexperience information for later communication. In some implementations,mobile device 102 collects and/or communicates the air interfaceinformation and the user experience information together. In someimplementations, mobile device 102 collects and/or communicates the airinterface information and the user experience information separately. Inthese implementations, the air interface information and the userexperience information may be collected and/or communicatedindependently of one another.

In some implementations, mobile device 102 communicates the airinterface information and the user experience information in response toa triggering event that indicates a particular quality of communicationbetween mobile device 102 and communications network 106. In someimplementations, the triggering event may include, without limitation,an indication from the user of the particular quality of communication,a determination by mobile device 102 of the particular quality ofcommunication based on the air interface information or the userexperience information, and/or other triggering event.

In some implementations, mobile device 102 may correlate the airinterface information and the user experience information andcommunicate the correlated air interface information and user experienceinformation. In other words, mobile device 102 may determine which airinterface information is collected or otherwise occurred when particularuser experience information is collected. In this manner, air interfaceinformation may be analyzed to determine whether actual networkperformance caused the network performance perceived at mobile device102.

In an example operation, mobile device 102 may collect air interfaceinformation exchanged with one or more base stations 104 ofcommunications network 106. Mobile device 102 may further collectparametric user experience information such as a signal strengthindication and/or subjective user experience information such as a userrating of communication quality. Mobile device 102 may periodicallycommunicate the air interface information and the user experienceinformation to server 110. These processes may execute on mobile device102 as background processes hidden from a user operating mobile device102.

The user may operate mobile device 102 by, without limitation, making aphone call, sending an email, receiving an email, browsing web pages,downloading content, uploading content, and/or performing otheroperations. During or after operating mobile device 102, the user mayinput the subjective user experience information. For example, the usermay initiate a user interface of mobile device 102 that allows the userto input the subjective user experience information. In this example,the user may wish to report poor quality and/or view a report thatdescribes reasons for the poor quality. In other implementations, mobiledevice 102 decides to initiate the user interface to prompt the user forthe subjective user experience information. In this example, mobiledevice 102 may pop up a dialog box that prompts the user to rate orotherwise answer questionnaires associated with the quality ofcommunication. In either case, mobile device 102 may store andperiodically communicate the parametric and/or subjective userexperience information to server 110.

Server 110 may receive the air interface information and the userexperience information from mobile device 102 and may determine whethercommunications network 106 caused the communication quality experiencedat mobile device 102 as indicated by the parametric user experienceinformation. For example, server 110 may determine whether thecommunications network 106 caused the signal strength indication basedon the air interface information. As would be appreciated based on thedisclosure herein, server 110 may determine whether communicationsnetwork 106 caused a plurality of communication qualities that occurredat different times at mobile device 102 as indicated by a correspondingplurality of parametric user experience information.

In implementations where server 110 receives air interface informationand/or user experience information periodically, server 110 may identifywhich user experience information to analyze based on theircorresponding qualities. In other words, server 110 may scan the userexperience information to determine which ones indicate a particularquality. If the particular quality is of interest, server 110 mayproceed to determine whether communications network 106 caused theparticular quality based on the air interface information. For example,server 110 may review a plurality of user experience information toidentify ones that indicate poor quality such as a low signal strengthindication or a low rating from a user that indicates the poor quality.

In implementations where server 110 receives air interface informationand/or user experience information based on triggers, server 110 mayanalyze the air interface information and/or user experience informationassociated with the triggers. For example, if a mobile device 102 or auser triggered communication of particular user experience information,server 110 may identify the particular user experience information foranalysis to determine whether communications network 106 caused thequality associated with the particular user experience information.

According to various implementations of the invention, server 110 mayinclude a processor 112, a memory 114, and/or other components thatfacilitate the functions of server 110 described herein. In someimplementations of the invention, processor 112 includes one or moreprocessors configured to perform various functions of server 110. Insome implementations of the invention, memory 114 includes one or moretangible (i.e., non-transitory) computer readable media. Memory 114 mayinclude one or more instructions that when executed by processor 112configure processor 112 to perform the functions of server 110.

FIG. 2 is a block diagram illustrating examples of components of aserver 240 and a mobile device 220 for user-experienced remotemonitoring of a communications network, according to variousimplementations of the invention. In some implementations, throughvarious components, mobile device 220 may collect air interfaceinformation and user experience information. In some implementations,mobile device 220 may upload either or both information to server 240.In some implementations, mobile device 220 uploads either or bothinformation periodically and/or based on a trigger that causes theupload. In some implementations, through various components, server 240may receive either or both information. In some implementations, server240 may correlate the air interface information and user experienceinformation with one another. In some implementations, server 240receives the air interface information and user experience informationalready correlated. In some implementations, server 240 determineswhether a communication quality indicated by user experience informationis due to an actual communication quality indicated by the air interfaceinformation.

According to various implementations of the invention, mobile device 220may include, for example, a user interface 222, a logging agent 224, anoperating system 226, a radio access network processor 228 (i.e., aradio network chipset), a processor 230, and a memory 232.

In some implementations, user interface 222 may be configured to receivesubjective user experience information from the user. In someimplementations, user interface 222 may be initiated by the user. Forexample, the user may wish to report and/or determine why poorcommunication quality (or other quality) occurred or is occurring atmobile device 220. In some implementations, user interface 222 may beinitiated by processor 230. In other words, mobile device 220 maydetermine poor communication quality occurred or is occurring and mayprompt the user for the user's opinion.

In some implementations, user interface 222 may present one or morequestionnaires that prompts for and receives input from the userregarding the communication quality. In some implementations, responsesto the questionnaires may include a multiple choice format and/or opencharacter input. In some implementations, user interface 222 may includea single input such as a “report poor quality” button. In someimplementations, user interface 222 may include a multi-input such as a“report poor quality” button, “report below average quality button,” andso forth.

In some implementations, logging agent 224 may determine parametric userexperience information. In some implementations, logging agent 224 maycollect the parametric user experience information from operating system226. For example, operating system 226 may include or otherwise publishAPIs that allow access to parametric user experience information (i.e.,basic network quality information) such as a signal strength indication.In these implementations, logging agent 224 may access the APIs tocollect the parametric user experience information.

In some implementations, logging agent 224 may determine air interfaceinformation by directly accessing radio access network processor 228.For example, a modified firmware for mobile device 220 may be suppliedto enable accessing radio access network processor 228. In someimplementations, radio access network processor 228 may include alogging function that is activated in order to make the air interfaceinformation accessible to logging agent 224. Radio access networkprocessor 228 may include one or more chipsets responsible forprocessing communication between mobile device 220 and a communicationsnetwork such as communications network 106 illustrated in FIG. 1. Forexample, radio access network processor 228 may process communicationwith a base station such as base station 104 illustrated in FIG. 1.

In some implementations, logging agent 224 may store the subjective userexperience information, the parametric user experience information,and/or the air interface information. In some implementations, loggingagent 224 may store the user experience information and/or the airinterface information in memory 232. In some implementations, loggingagent 224 may communicate one or more of the stored informationperiodically to a remote device such as to server 240. In someimplementations, logging agent 224 may communicate the subjective userexperience information, the parametric user experience information,and/or the air interface information as the information is collected. Insome implementations, logging agent 224 may communicate the subjectiveuser experience information, the parametric user experience information,and/or the air interface information in response to a trigger such as acommand from a user, a command from processor 230, or other trigger thatindicates one or more of the information should be communicated. In someimplementations, memory 232 includes one or more instructions that whenexecuted on processor 230 cause processor 230 to generate user interface222 and/or execute logging agent 224.

According to various implementations of the invention, server 240 mayinclude, for example, a user-client portal 242, a data portal 244, aclient instruction component 246, a report generator 248, a businessobjective module 250, a post processor 252, a memory 254, and a database260.

In some implementations, user-client portal 242 may be accessed by usersto view reports or other information associated with communicationquality determined or stored by server 240. For example, user-clientportal 242 may provide web pages, other interfaces, or data throughwhich users may access information available from server 240.

In some implementations, data portal 244 may receive parametric userexperience information, subjective user experience information, and/orair interface information (one or more of which may be collectivelyreferred to herein as “monitoring information”). For example, mobiledevice 220 may connect to data portal 244 or otherwise establish orreceive a communication channel with mobile device 220 to receive theinformation.

In some implementations, client instruction component 246 maycommunicate one or more instructions to be executed at mobile device220. In some implementations, the one or more instructions may includesoftware that configures processor 230 of mobile device 220 to executeuser interface 222. In some implementations, the one or moreinstructions may include software that configures processor 230 ofmobile device 220 to execute logging agent 224. In some implementations,the one or more instructions may include one or more questionnaires tobe presented to the user at mobile device 220. In some implementations,the one or more instructions may include one or more work orders bywhich mobile device 220 collects and/or uploads the monitoringinformation. For example, the one or more work orders may include,without limitation, an indication of: what types of the information tocollect (such as signal strength, types of user feedback, types of airinterface information), how often/when to collect the information, howoften/when to upload the information, how often/when to present thequestionnaires, and/or other instructions for remote monitoring of userexperience and/or network information.

In some implementations, business objectives module 250 may include oneor more descriptions of the business objectives of remote monitoring. Insome implementations, the business objectives may affect the types ofinstructions communicated to mobile device 220, how/what data is handledat data portal 244, post-processing of data received at data portal 244,and/or other configurations that affect operation of server 240 and/ormobile device 220 when monitoring user experience information and airinterface information. In some implementations, the business objectivesmay include an indication that “poor” or other quality indications fromusers are to be investigated by post-processor 252. In someimplementations, the business objectives may include an indication ofhow often user experience information and/or air interface informationshould be collected and/or uploaded. In some implementations, thebusiness objectives may include an indication of what geographiclocations (as indicated by a base station identifier, for example)should be monitored.

In some implementations, post-processor 252 may process data received atdata portal 244. In some implementations, post-processor 252 maycorrelate different data sources with one another. In this mannerpost-processor 252 may refer to appropriate data during processing. Forexample, in some implementations, post-processor 252 may correlate theuser experience information and the air interface information. In thismanner, air interface information may be used to determine whether anactual quality of a communications network caused the perceived qualityindicated by the user experience information. In some implementations,post-processor 252 may correlate business objectives with incoming data(such as the monitoring information). In some implementations,post-processor 252 may identify the mobile device from which theinformation originated. In this manner, a particular mobile device maybe monitored. In some implementations, post-processor 252 may identifythe location from which the information originated. In this manner, acommunications network may be monitored based on location. In someimplementations, post-processor 252 may determine whether qualityindicated by the user experience information is caused by the actualquality indicated by the air interface information. In someimplementations, post-processor 252 may store the correlatedinformation, identification information, location information, and/orother information. In some implementations, post-processor 252 may storethe information using database 260. In some implementations,post-processor 252 may group information based on user, devicegroupings, and/or other groupings.

In some implementations, report generator 248 may use the post-processedinformation from post-processor 252 and generate reports. In someimplementations, the reports display the user experience informationcorrelated with the air interface information. For example, a “poorsignal indication” by a particular mobile device at a particular timemay be displayed alongside air interface information at or near theparticular time. In this manner, based on the report, whether thecommunications network was responsible for the poor signal indicationmay be determined. In some implementations, the reports may provideindications of the types of information being displayed, such as whetherthe user experience information is parametric user experienceinformation and/or subjective user experience information. In someimplementations, report generator 248 may store the reports in a memorysuch as database 260.

In some implementations, the reports are accessible by users of mobiledevices 220 interested in knowing why quality of network communicationoccurred. For example, a user may use user-client portal 242 in order toaccess web-based reports of the performance of mobile device 220. Basedon an identity of the user and/or mobile device, for example, the usermay be granted access to the reports for mobile device 220 and/or thecommunications network. In some implementations, the reports areaccessible by users such as network system administratorstroubleshooting or optimizing network configurations. In theseimplementations, users may view reports that associate subjective userexperience information and air interface information on a location,regional, or other macro-level view. In some implementations, users mayview reports based on a micro-level (i.e., reports for individual mobiledevices).

FIG. 3 is a flow diagram illustrating an example process among a mobiledevice and a server device for user-experienced remote monitoring of acommunications network, according to various implementations of theinvention. The various processing operations and/or data flows depictedin FIG. 3 (and in the other drawing figures) are described in greaterdetail herein. The described operations for a flow diagram may beaccomplished using some or all of the system components described indetail above and, in some implementations of the invention, variousoperations may be performed in different sequences. According to variousimplementations of the invention, additional operations may be performedalong with some or all of the operations shown in the depicted flowdiagrams. In yet other implementations, one or more operations may beperformed simultaneously. Accordingly, the operations as illustrated(and described in greater detail below) are examples by nature and, assuch, should not be viewed as limiting.

In an operation 302, process 300 may register a new user. For example,process 300 may receive, from a user, basic user information such as auser name, a user address, a signed (electronically or otherwise)license agreement, a mobile device identifier, and/or other userinformation.

When the new user is registered, in an operation 304, process 300 mayinstall an agent onto a mobile device of a user. The agent may includeprogramming that causes the mobile device to generate a user interfacesuch as user interface 222 illustrated in FIG. 2, execute a loggingagent such as logging agent 224 illustrated in FIG. 2, and/or performother suitable functions. In some implementations, while installing anagent in operation 302, process 300 may provide instructions thatinclude settings by which the agent should operate. For example, theinstructions may include: work orders that define when, where and/or howto collect and upload monitoring information, the one or morequestionnaires, and/or other settings. When the agent is installed,process 300 may register the mobile device for user experienced remotemonitoring.

In an operation 306, process 300 may collect and upload monitoringinformation. In some implementations, process 300 collects and uploadsthe monitoring information according to work orders or other settingsreceived during device registration.

In an operation 308, process 300 may receive the collected and uploadedmonitoring information. In some implementations where user experienceinformation and air interface information is received, process 300 maycorrelate both. In some implementations, the monitoring information isstored for later analysis.

In an operation 310, process 300 may generate one or more reports. Thereports may be specific to a particular mobile device, specific to aparticular user who may operate multiple devices, specific to aparticular location within a communications network (such as certainbase stations or geographic locations of the communications network),network-wide, and/or other grouping as appropriate.

FIG. 4 is a flow diagram illustrating an example of a process ofuser-experienced remote monitoring of a communications network at aserver, according to various implementations of the invention. In anoperation 402, process 400 may receive air interface information from amobile device. The air interface information may originate from signalscommunicated between a mobile device and the communications network andmay indicate a communication quality between the mobile device and thecommunications network. In an operation 404, process 400 may receiveuser experience information. The user experience information mayindicate a quality of performance experienced at the mobile device. Inan operation 406, process 400 may determine a quality of performance atthe mobile device based on the user experience information. In anoperation 408, process 400 may determine whether the quality ofperformance occurred due to network quality based on the air interfaceinformation.

FIG. 5 is a flow diagram illustrating an example of a process ofuser-experienced remote monitoring of a communications network at amobile device, according to various implementations of the invention. Inan operation 502, process 500 may access one or more chipsets of themobile device. In an operation 504, process 500 may collect airinterface information from signals communicated between the mobiledevice and the communications network via the accessed one or morechipsets. The air interface information may indicate performance of thecommunications network. In an operation 506, process 500 may determineuser experience information. The user experience information mayindicate a measured or perceived quality of performance experienced atthe mobile device. In an operation 508, process 500 may communicate theair interface information and the user experience information to aremote device. The air interface information and the user experienceinformation may be used to correlate whether communication quality atthe mobile device is caused by the communications network.

FIG. 6 is a flow diagram illustrating an example of a process ofuser-experienced remote monitoring of a communications network,according to various implementations of the invention.

In an operation 602, process 600 may read work order instructions. Forexample, process 600 may read the setting and instructions that a systemadministrator has defined. The instructions may indicate a type ofmonitoring information to collect/upload, whether measurements should beactive such as during a call or data transmission, and when to uploadthe data (based on a trigger or periodically).

In an operation 604, process 600 may determine whether to perform basiclogging. If basic logging is specified by the work order instructions,process 600 may proceed to an operation 606, where parametric userexperience information (basic information) is logged. In someimplementations, the basic information is accessed from an API of theoperating system of a mobile device. Process 600 may proceed to anoperation 610, where a determination of whether continuous (periodic)logging is enabled according to the work order instructions.

Returning to operation 604, if basic logging is not specified, process600 may proceed to an operation 608, where air interface information(detailed information) is logged. In operation 608, process 600 maydirectly connect to one or more chipsets of a mobile device in order toaccess and collect signaling between the mobile device and acommunications network such as a cellular network. In someimplementations, the air interface information may be temporarily storedfor later analysis and/or upload. Upon logging the detailed information,process 600 may proceed to operation 610.

If in operation 610 continuous logging is specified, process 600 mayproceed to an operation 616, where the data is uploaded. If continuouslogging is not specified process 600 may proceed to an operation 612,where a determination whether upload was triggered by a user is made. Ifthe upload was not triggered by the user, process 600 may proceed tooperation 616, where the data is uploaded. If the upload was triggeredby the user, process 600 may proceed to operation 614, where subjectiveuser experience information is retrieved. For example, process 600 maycause a questionnaire to be presented to the user. The questionnaire mayprompt input of the quality such as: “How did you rate the audio qualityof your last call?” and “What was your experience of the last streamingvideo session?” In this manner, user assessment/opinion of thecommunication quality may be logged. Once the subjective user experienceinformation is logged, process 600 may proceed to operation 616.

When data is uploaded in operation 616, process 600 may proceed tooperation 618, where a determination whether the upload failed is made.If the upload failed, process 600 may return to operation 616, where theupload is re-attempted. If the upload did not fail, process 600 mayproceed to an operation 620, where a determination whether to continuemonitoring is made. If monitoring is to continue, process 600 may returnto operation 602, where work order instructions are read. If monitoringis not to continue, process 600 may be suspended or terminate in anoperation 622.

Implementations of the invention may be made in hardware, firmware,software, or any suitable combination thereof. Implementations of theinvention may also be implemented as instructions stored on a machinereadable medium, which may be read and executed by one or moreprocessors. A tangible machine-readable medium may include any tangible,non-transitory, mechanism for storing or transmitting information in aform readable by a machine (e.g., a computing device). For example, atangible machine-readable storage medium may include read only memory,random access memory, magnetic disk storage media, optical storagemedia, flash memory devices, and other tangible storage media. Further,firmware, software, routines, or instructions may be described in theabove disclosure in terms of specific exemplary implementations of theinvention, and performing certain actions. However, it will be apparentthat such descriptions are merely for convenience and that such actionsin fact result from computing devices, processors, controllers, or otherdevices executing the firmware, software, routines, or instructions.

Implementations of the invention may be described as including aparticular feature, structure, or characteristic, but every aspect orimplementation may not necessarily include the particular feature,structure, or characteristic. Further, when a particular feature,structure, or characteristic is described in connection with an aspector implementation, it will be understood that such feature, structure,or characteristic may be included in connection with otherimplementations, whether or not explicitly described. Thus, variouschanges and modifications may be made to the provided descriptionwithout departing from the scope or spirit of the invention. As such,the specification and drawings should be regarded as exemplary only, andthe scope of the invention to be determined solely by the appendedclaims.

The invention claimed is:
 1. A computer-implemented method ofuser-experienced remote monitoring of a communications network,comprising: receiving, by a server device, air interface informationfrom a mobile device, wherein the air interface information originatesfrom signals communicated between the mobile device and thecommunications network, wherein the air interface information iscollected via one or more radio access network chipsets of the mobiledevice, and wherein the air interface information indicates an actualcommunication quality between the mobile device and the communicationsnetwork; determining, by the server device, user experience information,wherein the user experience information indicates a perceivedcommunication quality at the mobile device; associating, by the serverdevice, the air interface information and the user experienceinformation with one another; and determining, by the server device,whether the perceived communication quality is due to the actualcommunication quality based on the associated air interface informationand user experience information.
 2. The computer-implemented method ofclaim 1, wherein receiving the air interface information comprisesreceiving one or more of an air interface protocol, a message orparameter reported to and from the mobile device and the communicationsnetwork to control network traffic, or a network configuration.
 3. Thecomputer-implemented method of claim 1, wherein determining userexperience information comprises receiving parametric user experienceinformation that indicates the perceived communication quality, andwherein the parametric user experience information is accessed via anoperating system of the mobile device.
 4. The computer-implementedmethod of claim 3, wherein receiving parametric user experienceinformation comprises receiving one or more of a signal strengthindicator or a battery life.
 5. The computer-implemented method of claim1, wherein determining user experience information comprises receivingsubjective user experience information that indicates the perceivedcommunication quality, and wherein the subjective user experienceinformation originates from a user of the mobile device.
 6. Thecomputer-implemented method of claim 5, wherein receiving subjectiveuser experience information comprises receiving one or more of a userindication of the perceived communication quality, one or more ratingsby the user that indicates the quality of performance, or a response toa questionnaire.
 7. The computer-implemented method of claim 1, furthercomprising: identifying, by the server device, the air interfaceinformation to analyze based on the user experience information.
 8. Acomputer-implemented method of user-experienced remote monitoring of acommunications network, comprising: accessing, by a mobile device, oneor more radio access network chipsets of the mobile device; collecting,by the mobile device, air interface information from signalscommunicated between the mobile device and the communications networkvia the accessed one or more radio access network chipsets, wherein theair interface information indicates an actual communication quality;determining, by the mobile device, user experience information, whereinthe user experience information indicates a perceived communicationquality at the mobile device; and communicating, by the mobile device,the air interface information and the user experience information to aremote device, wherein the air interface information and the userexperience information are used to determine whether the perceivedcommunication quality is caused by the actual communication quality. 9.The method of claim 8, wherein collecting the air interface informationcomprises collecting one or more of an air interface protocol, a messageor parameter reported to and from the mobile device and thecommunications network to control network traffic, or a networkconfiguration.
 10. The method of claim 8, wherein the user experienceinformation comprises parametric user experience information thatindicates the perceived communication quality, and wherein determininguser experience information comprises accessing an operating system ofthe mobile device and collecting the parametric user experienceinformation via the accessed operating system.
 11. The method of claim10, wherein collecting the parametric user experience informationcomprises collecting one or more of a signal strength indicator, or abattery life.
 12. The method of claim 8, wherein the user experienceinformation comprises subjective user experience information thatindicates the perceived communication quality, and wherein determininguser experience information comprises: receiving, by the mobile devicefrom the user, an indication of a quality of communication experiencedat the mobile device.
 13. The method of claim 12, wherein the userexperience information comprises one or more user ratings of the qualityof performance, the method further comprising: prompting, by the mobiledevice in response to the indication from the user, input of the one ormore user ratings; and receiving, by the mobile device, the one or moreuser ratings.
 14. The method of claim 8, wherein the air interfaceinformation or the user experience information are communicatedperiodically at intervals.
 15. The method of claim 8, wherein the airinterface information or the user experience information arecommunicated in response to a triggering event that indicates aparticular quality of communication between the mobile device and thecommunications network.
 16. The method of claim 15, wherein thetriggering event comprises: receiving, by the mobile device from a userof the mobile device, an indication of the particular quality ofcommunication.
 17. The method of claim 15, wherein the triggering eventcomprises: determining, by the mobile device, the perceivedcommunication quality based on the air interface information or the userexperience information.
 18. The method of claim 8, wherein communicatingthe air interface information and the user experience informationfurther comprising: correlating the air interface information and theuser experience information and communicating the correlated airinterface information and user experience information.
 19. The method ofclaim 8, wherein collecting air interface information occurs during aproduction operation of the mobile device.
 20. The method of claim 8,wherein collecting air interface information occurs during a testingoperation of the mobile device.
 21. A system of user-experienced remotemonitoring of a communications network, comprising one or more servers,the one or more servers comprising: at least one processor configuredto: receive air interface information from a mobile device, wherein theair interface information originates from signals communicated betweenthe mobile device and the communications network, wherein the airinterface information is collected via one or more radio access networkchipsets of the mobile device, and wherein the air interface informationindicates an actual communication quality between the mobile device andthe communications network; determine user experience information,wherein the user experience information indicates a perceivedcommunication quality at the mobile device; associate the air interfaceinformation and the user experience information with one another; anddetermine whether the perceived communication quality is due to theactual communication quality based on the associated air interfaceinformation and user experience information.
 22. The system of claim 21,wherein when configured to receive the air interface information, theone or more processors further configured to receive one or more of anair interface protocol, a message or parameter reported to and from themobile device and the communications network to control network traffic,or a network configuration.
 23. The system of claim 21, wherein whenconfigured to determine user experience information, the one or moreprocessors further configured to: receive parametric user experienceinformation that indicates the perceived communication quality, andwherein the parametric user experience information is accessed via anoperating system of the mobile device.
 24. The system of claim 23,wherein when configured to receive parametric user experienceinformation , the one or more processors further configured to: receiveone or more of a signal strength indicator, or a battery life.
 25. Thesystem of claim 21, wherein when configured to receive user experienceinformation, the one or more processors further configured to: receivesubjective user experience information that indicates the perceivedcommunication quality, and wherein the subjective user experienceinformation originates from a user of the mobile device.
 26. The systemof claim 25, wherein when configured to receive subjective userexperience information, the one or more processors further configuredto: receive one or more of a user indication of the perceivedcommunication quality, one or more ratings by the user that indicatesthe quality of performance, or a response to a questionnaire.
 27. Thecomputer-implemented method of claim 21, the one or more processorsfurther configured to: identify the air interface information to analyzebased on the user experience information.
 28. The system of claim 21,wherein when configured to determine user experience information, theone or more processors further configured to: determine parametric userexperience information based on the air interface information.
 29. Thesystem of claim 21, wherein when configured to determine user experienceinformation, the one or more processors further configured to: receivefirst parametric user experience information and determine secondparametric user experience information based on the first parametricuser experience information.